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Executive Summary 


Students who are deaf or hard of hearing (Deaf/HH) and communieate using Ameriean Sign Lan- 
guage (ASL) may benefit from sign aeeommodations on state assessments. However, there have been 
challenges in standardizing assessment content including how items should be signed. Technology- 
enhanced assessment offers the opportunity to provide standardized sign support delivered through the 
test platform. One goal of the Guidelines for Accessible Assessment Project (GAAP) was to develop 
research-based sign guidelines that can be used across states, consortia, and assessment vendors to 
produce reliable and valid signed representations of assessment items and tasks for students who 
communicate using sign language. 

This report highlights the findings from cognitive labs conducted with students to gain their input 
and feedback on the proposed guidelines. A total of 46 elementary-, middle-, and high- school- aged 
students who were Deaf/HH and communicated using ASL participated in cognitive labs. 

The research findings point to the students’ preferences for the provision of the sign support in general 
and for this support being grounded in ASL rather than representations aligned with other languages. 
Generally, students also voiced their preferences for graphics to be described in detailed ASL. However, 
overall research results were mixed. These findings support the approach of making individualized 
decisions informed by students’ unique needs and preferences. Additionally, the study results suggest 
that in the process of developing sign supports for computer-based assessments, it is important to 
consider the individual needs of the diverse community of Deaf/HH students. 
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Introduction 


The Elementary and Secondary Education Act (ESEA) and the Individuals with Disabilities 
Education Improvement Act (IDEA) both require that accommodations be provided, as appropri- 
ate, to those students with disabilities who need them during assessments. If students’ individual 
accessibility and accommodations needs are not addressed appropriately, such students are placed 
at a severe disadvantage in demonstrating what they know and can do on state assessments. 

In the context of the Common Core State Standards (CCSS), assessment consortia such as the 
Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter 
Balanced Assessment Consortium (Smarter Balanced) have prepared guidelines regarding the 
roles and responsibilities of individuals who assist in the administration of accommodations. 
However, these guidelines often require additional detail to provide valid and equitable support 
for students who need sign-based accommodations. 

The Guidelines for Accessible Assessment Project (GAAP) that provided data for this report 
was carried out to meet the following goals: 

1. to develop research-based audio guidelines that can be used across states, consortia, and 
assessment vendors to produce reliable and valid audio representations of assessment items 
and tasks for students with low vision or reading-based learning disabilities, and students 
who are English language learners (EEEs), and 

2. to develop research-based sign guidelines that can be used across states, consortia, and as- 
sessment vendors to produce reliable and valid signed representations of assessment items 
and tasks for students who communicate using sign language. 

The project was conducted to inform the development of statewide assessments that are more 
accessible than previous versions to students with diverse characteristics. Data collected through 
student cognitive labs have implications for developing accessible tests for students with a 
variety of assessment needs, and this project resulted in a number of research efforts to serve 
these students. Two such efforts were designed to produce guidelines for accessible assessments 
using sign and audio supports. This report highlights evidence collected to address the second 
goal of the project — the development of research-based sign guidelines for use on statewide 
assessments. The following is the research question addressed in the report: 

• What sign accommodations are the most usable and most effective for students to use on 
assessments? 

This question was addressed by gathering input from state education agency personnel, includ- 
ing assessment and content experts. In addition feedback was gathered from students. The data 
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collected focus on how assessment content should be signed for students who are deaf or hard 
of hearing (Deaf/HH) and communicate using American Sign Language (ASL). 


Method 

Cognitive labs were conducted in four states that were geographically dispersed, with diverse 
characteristics. These states included PARCC states, Smarter Balanced states, and non-consor- 
tium affiliated states. 

A total of 46 elementary-, middle-, and high-school-aged students who were Deaf/HH and com- 
municated using ASL participated in a cognitive lab (Table 1). Overall, 16 elementary school 
students, 14 middle school students, and 16 high school students participated in the study. As 
shown in the table, 23 students were male, and 23 students were female. The table also shows 
the grade distribution of the student sample with the highest number of participants in the fourth, 
sixth, and eleventh grades (n=6 at each grade level), and the lowest number of students in the 
tenth grade (n=l). 

Table 1 . Student characteristics 


Characteristics 

Number 

Gender 


Male 

23 

Female 

23 

Grade 


3rd 

5 

4th 

6 

5th 

5 

6th 

6 

7th 

3 

8th 

5 

gth 

5 

10* 

1 

11* 

6 

12* 

4 


Each student was provided a brief test with isomorph pairs of items, that is, item sets in which 
one item was slightly different from the other. During the assessment, two researchers observed 
students and took notes on students’ interaction with the items. See Appendix A for the cogni- 
tive lab instruments used in the study for the elementary, middle, and high school assessments. 
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The researchers were interested in how engaged with items students were when responding to 
test items. Students were observed to not answer test items earnestly if they reported that they 
guessed the answer or if they responded to the item pair quickly, selecting the same answer 
choice for both items. In addition, the researchers noted how many times the student replayed 
the American Sign Language (ASL) video for each item. 

Following the completion of each item pair, students were asked several questions about each 
isomorph in the pair of items, including item difficulty, representation preference, and what 
aspects of the accommodated items were helpful or not helpful. 


Elementary School Assessment With Sign Support 

The elementary school assessment with sign support included six questions regarding effective- 
ness of items and explored: 

• whether coordinate grids contained in items should be represented in ASL in specific (on a 
signer’s hand) or general (in front of a signer’s body) space; 

• whether items should be presented in ASL from the perspective of a character in the item 
or from the perspective of the computer screen; 

• whether expressions and equations should be presented in ASL; 

• whether key words that have signs that might not be common to students or are directly 
related to the construct being measured should be signed only or signed and finger-spelled; 

• whether items containing a repetitive action should be presented in ASL using singular 
signing or plural signing; and 

• whether large numbers within items should be signed. 

Each of the six issues is discussed below. A total of 16 elementary school students participated 
in the study. Most students were observed to respond to the test items earnestly. In several cases, 
students either were observed to guess item responses or admitted to guessing themselves. 

Study findings indicate that most elementary school students reported that viewing ASL videos 
helped them understand test questions better. One student stated that “Signs help me understand.” 
Another student pointed out that, “Sometimes, I’m uncertain when reading, so videos help.” 
One student, when responding to the question about helpfulness of ASL videos, also commented 
on their length, “Yes, but maybe less time consuming. Explanations were long, I prefer shorter 
clips. Some things could be explained differently, I didn’t understand some of them.” 
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Issue 1 : Do students prefer that coordinate grids contained in items be 
represented in ASL in specific (on a signer’s hand) or generai (in front of a 
signer’s body) space? 

Two items were created to answer whether students who are Deaf/HH prefer to have a signer 
present specific space on a coordinate grid (on a signer’s hand) in Item 1 or represent the space 
(in front of a signer’s body) in Item 2. Table 2 summarizes elementary school students’ responses 
to Item Set 1 . 

Table 2. Student responses to questions about Item Set 1 


Types of Responses 

Item 1 
(Specific 
Space) 

Item 2 
(General 
Space) 

Replaying Video 

3 

1 

Difficulty 



Easy 

7 

8 

Medium 

0 

0 

Difficult 

5 

4 

No Reported Difficulty 

4 

4 

Item Preference 

8 

8 


Replaying Video 

The majority of students did not replay the video for either item in Item Set 1 . However, one 
student replayed both items in Item Set 1 . Two additional students replayed the video for only 
the first item, where the grid was presented in specific space. 

Reported Difficulty 

More students perceived the two test items to be easy (n=7 and n=8 respectively) than difficult 
(n=5 and n=4 respectively). No item difficulty was reported in four instances. 

Item Preferences 

The students who completed Item Set 1 were evenly split as to whether they preferred the first or 
second item. Students who preferred the first item were either unsure of why they preferred Item 

1 , or stated that they found that the signing was easier to follow and see than the video in Item 

2. Students who preferred Item 2 thought the video was clearer than in Item 1, and stated that it 
was difficult to see the numbers that were signed for Item 1 because they were being signed on 
the signer’s hand. One student pointed out, “The second item was more helpful — the signer set 
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up the graph better than in the first one. The graph wasn’t helpful on his hand.” Another student 
noted, “Better with the grid in front of him — I could see better and I could understand the words.” 

Issue 2: Do students prefer that items be presented in ASL from the 
perspective of a character in the item or from the perspective of the computer 
screen? 

Item Set 2 was developed to answer from what perspective students prefer an item be explained — 
a character in the item or from the perspective of a computer screen. In the first item, the question 
about planting soybeans in a drag-and-drop grid was explained from the character’s perspective. 
The second item was explained from the perspective of the computer screen. Table 3 summarizes 
the findings for Item Set 2. 


Table 3. Student responses to questions about Item Set 2 


Types of Responses 

Item 3 
(Character 
Perspective) 

Item 4 
(Computer 
Screen 
Perspective) 

Replaying Video 

0 

0 

Difficulty 



Easy 

7 

8 

Medium 

1 

1 

Difficult 

4 

3 

No Reported Difficulty 

4 

4 

Item Preference 

5 

9 


Note: Two students reported having no preference for either item in Item Set 2. 


Replaying Video 

No students replayed the video for Item Set 2. One student read questions before playing the 
video, and another student began answering questions while the video was playing. 

Reported Difficulty 

Of those students who reported item difficulty, the majority thought the item set was easy. Four 
students reported the first item to be difficult, and three students reported that the second item 
was difficult. Four students did not report an item difficulty level for this item set. 
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Item Preferences 


For Item Set 2, students were slightly more likely to prefer Item 4 (n=9), where the question was 
signed from the perspective of the computer screen, than Item 3 (n=5) where the question was 
signed from the perspective of the character in the item. There were two students who indicated 
that they had no preference for the items. One student who preferred the first item stated, “I’m 
the farmer and get to plant it — it’s easy.” Others also commented that they could understand 
more when the item was explained from the farmer’s perspective, or that they could clearly 
see the signing in the first item. Students who preferred the second item stated that they liked 
that the signing matched what was on the screen or stated that the use of space was more in 
line with the item: “I could see clearly. He wasn’t signing English, but ASL. I prefer ASL. ... I 
like drag-and-drop — I like putting things in place.” Another student stated, “The item about the 
com was more clear to me.” One of the students who stated no preference found both items to 
be confusing and difficult to understand, while the other student with no preference liked both 
videos and believed that the items were easy. 

Issue 3: Do students prefer that expressions and equations be presented in 
ASL? 

To answer whether students prefer that expressions and equations be presented in ASL, an item 
set was developed. In the first item, the expressions and equations were signed in ASL, while in 
the second item, the expressions and equations were referred to but not signed for the students. 
Table 4 summarizes student responses to questions about Item Set 3. 


Table 4. Student responses to questions about Item Set 3 


Types of Responses 

Item 5 
(Equations 
not Signed) 

Item 6 
(Equations 
Signed) 

Replaying Video 

0 

0 

Difficulty 



Easy 

4 

6 

Medium 

2 

2 

Difficult 

4 

3 

No Reported Difficulty 

6 

5 

Item Preference 

3 

9 


Note: Two students reported having no preference for either item in Item Set 3. Two students did not answer the 
items in Item Set 3. 
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Replaying Video 


As shown in the table, no students replayed the video for either item in Item Set 3. There were 
two students that read and answered the items first before watching the video. One student 
explained, “I skipped the videos because I knew the meaning of the words in the question.” 

Reported Difficulty 

Students seemed to have a variety of perceptions regarding the item difficulty, with two students 
considering the item pair to be of medium difficulty, and others perceiving the items to be either 
easy or difficult. One student who perceived the items to be difficult noted that these were “hard 
questions. I haven’t learned the content yet. The teacher said this material is coming down the 
road.” No item difficulty was reported in six cases for Item 5 and five cases for Item 6. 

Item Preferences 

The majority of students who completed Item Set 3 stated that they preferred Item 6 (n=9). The 
two students who preferred Item 5 did not mention why they preferred the first item. The three 
students who preferred Item 6 stated that the signer was more efficient, and easier to follow 
than in the first item. Students said “I couldn’t follow the first one,” and stated that the second 
item “was easier to understand,” because of the concise signing. One of the students also noted, 
“The second item was clearer — it actually said what the choices were.” The student who did 
not endorse a preference was not able to discern the difference between the signing of the two 
items without prompting of the researchers, and stated that “it really didn’t matter,” how the 
content was signed. 

One student also commented on the speed of signing: “The signer was really fast, I liked how 
he signed it.” Two other students brought up the issue of sign variations in ASL. One of them 
stated, “I didn’t understand the signs for multiplying and dividing. He should have used a dif- 
ferent sign [using two fingers].” Another student pointed out that “The signer should have used 
the sign for multiplication instead of an ‘x.’” 

Issue 4: Do students prefer that key words that have signs that might not be 
common to students or are directiy reiated to the construct being measured be 
signed oniy or signed and finger-speiied? 

To answer the question posed by Issue 4, an isomorph pair of items was developed. In the first 
item, the signer only used ASL signs for key words, while in the second item, the signer used 
ASL signs for key words and finger-spelled them. Student responses for Item Set 4 are reported 
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in Table 5. One student had not learned how to multiply or divide, and did not provide answers 
for this item set. 


Table 5. Student responses to questions about Item Set 4 


Types of Responses 

Item 7 
(ASL Sign 
Only) 

Item 8 
(ASL Sign 
and Finger- 
spelling) 

Replaying Video 

1 

1 

Difficulty 



Easy 

4 

7 

Medium 

0 

0 

Difficult 

3 

0 

No Reported Difficulty 

9 

9 

Item Preference 

7 

6 


Note: Two students reported having no preference for either item in Item Set 4. One student did not answer the 
items in Item Set 4. 

Replaying Video 

For Item Set 4, one students replayed videos from each item. One student did not watch the 
videos for this set, “I did not watch the videos. I thought he was not clear in videos, so decided 
it was easier to just read the problems.” 

Reported Difficulty 

Most students found the two items were easy (particularly the second one), although in three 
instances, the first item was reported to be difficult. In nine cases, no difficulty was reported 
for this item set. 

Item Preferences 

For Item Set 4, students were only slightly more likely to prefer Item 7 (n=7), where the key 
word was presented using the ASL sign only, and unfamiliar words specific to the content were 
not finger-spelled. Students who preferred Item 7 were typically familiar with the signs in the 
items, and reported that they did not need them finger-spelled. Additionally, one student stated 
that “not all people know finger-spelling, but they know ASL” when explaining why the student 
preferred the first item. Another student stated, “I knew the signs. It was confusing to have the 
finger- spelling.” Still another student who preferred Item 7 reported not understanding finger- 
spelling. 
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Students who preferred Item 8 (with finger-spelling) indicated that they knew many of the signs 
(e.g., right angle, symmetry, length) but that the finger- spelling was still helpful. Some students 
addressed sign varieties in ASL. One of them noted, “They sign ‘right angle’ differently in my 
class. But I still knew the sign when the signer did it. It was helpful to have ‘right angle,’ ‘sym- 
metry,’ and ‘length’ finger-spelled.” In a parallel statement, another student reported, “The signs 
that the interpreter uses are different from what I am used to. In class, we sign right angles by 
moving angles around the page. He did not do that.” One student also reported knowing some 
signs but not others: “I know the sign for ‘square’ but not for ‘right angle.’ I don’t know the sign 
for ‘length.’ I also don’t know what ‘symmetry’ means.” 

Issue 5: Do students prefer that items containing a repetitive action be 
presented in ASL using singuiar signing or piurai signing? 

An item set was developed to answer the research question of whether students prefer repeti- 
tive actions in ASL be represented using singular or plural signing. The first item presented the 
question using singular signing when the item stated that a student had to put 60 muffins into 
six boxes, asking the students to determine how many muffins were in each box. The second 
item was based on a similar question (i.e., cookies and bags) but the action was presented us- 
ing plural signing. Table 6 presents data on students’ responses to questions about Item Set 5. 
Two students did not respond to this item set, bringing the total number of students who were 
Deaf/HH down to 14. 

Table 6. Student responses to questions about Item Set 5 


Types of Responses 

Item 9 
(Singular 
Sign) 

Item 10 
(Plural Sign) 

Replaying Video 

0 

0 

Difficulty 



Easy 

7 

7 

Medium 

0 

0 

Difficult 

0 

3 

No Reported Difficulty 

9 

6 

Item Preference 

7 

7 


Note: Two students did not answer the items in item Set 5. 


Replaying Video 

There were no students who replayed the signing video for Items 9 and 10. However, two 
students either answered the question without watching the videos or watched the video after 
reading the problem and selecting an answer. 


NCEO 


9 







Reported Difficulty 


The seven students who reported on the difficulty of these two items considered them to be easy. 
Three students also thought that the second item was difficult. 

Item Preferences 

Students who responded to the preference question were equally split according to which item 
they preferred. Students who preferred the first item, stated that they understood the term “box” 
(i.e., dividing muffins into boxes) more than they understood the second item (where cookies 
were placed in bags). Though none of the students mentioned the use of plural or singular signing 
in their determination of item preference, a few students stated that it was easier to understand 
the signer in the first item (singular signing) than the second item (plural signing). Students who 
preferred the second item believed that the signing was clearer, and that the question was easier 
to understand than in the first item. According to one student, “The second item was clearer 
than the first one — it included bigger, clearer signs. Finger-spelling was helpful. I did not like 
the signs close to the interpreter’s body.” Another student noted, “I can understand questions 
better when they are signed, not finger-spelled.” Generally, students from both preferences ap- 
preciated having “box” and “bag” finger-spelled. 

Issue 6: Do students prefer that large numbers within items be signed? 

An item set was developed to determine whether students preferred that large numbers be signed 
or simply referred to. In the first item, large numbers (e.g., 336, 154) in the item and answer 
choices were signed for students. In a second item, similar numbers were mentioned, but they 
were not explicitly signed. Student responses to questions about Item Set 6 are summarized in 
Table 7. 

Table 7. Student responses to questions about Item Set 6 


Types of Responses 

Item 11 
(Large 
Numbers 
Signed) 

Item 12 
(Large 

Numbers not 
Signed) 

Replaying Video 

1 

0 

Difficulty 



Easy 

3 

4 

Medium 

2 

2 

Difficult 

1 

0 

No Reported Difficulty 

10 

10 

Item Preference 

6 

5 


Note: Two students did not answer the items in item Set 6. Three students did not report a preference for either 
item in Item Set 6. 
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Replaying Video 


There were no students who replayed the video for Items 1 1 and 12. Two students skipped one 
or both videos and simply answered the questions, or began answering the question while the 
video was still playing. 

Reported Difficulty 

A total of six students responded to the question about item difficulty. Of those six, the students 
were primarily split on whether they thought the items were easy or of medium difficulty. One 
student reported the first item to be difficult. The students who believed that the items were of 
medium difficulty stated that it was unclear how certain words were signed. 

Item Preferences 

Study findings indicate that students were slightly more likely to prefer Item 11, where the 
large numbers in the item were signed, than Item 12, where the large numbers were not signed. 
Students who preferred the first item said that the item was signed more clearly, and that signing 
the numbers was helpful for students who struggle to read. Students who preferred the second 
item stated that the second item was “clearer” or that they could simply “understand it better.” 
Regardless of preference for the signing of numbers, some students seemed to like that “pages” 
and “miles” (used in the items) were finger-spelled to clarify the words, though one student 
mentioned that rather than finger- spelling “miles,” a reference to the distance measure would 
have been more helpful. One student seemed to prefer both ways of term presentation, “I kind 
of like both - finger-spelled and signed.” Some students noted that they did not need to have 
the terms finger- spelled. 

When responding to the question about the preference for finger-spelling, one student noted, 
“I just prefer the sign. When it was finger-spelled, I totally missed it, I didn’t get it. He finger- 
spelled it quickly, I had a hard time understanding what he was saying.” Another student stated: 
“I didn’t need to have ‘pages’ finger- spelled. Also I didn’t need to have the answer signed — just 
wanted to go ahead and click on an answer choice without waiting for the video to end.” Ac- 
cording to yet another student, “no finger-spelling was more helpful. I didn’t understand the 
finger-spelling, and I knew the three terms.” 


Middle School Assessment With Sign Support 

The middle school assessment covered the following five questions regarding effectiveness of 
items. These questions focused on: 
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• whether items containing a lot of information prior to the question should be signed using 
the English text order (no-diamond) or by presenting the question before the item informa- 
tion and then again after the item information (diamond); 

• whether information on number lines should be presented by focusing on zero or by focus- 
ing on the number in the center of the number line; 

• whether in items that require comparing expressions, the signer should refer back to the 
original expression or should not refer back to the original expression; 

• whether items should be presented in ASL from the signer’s perspective of a diagram con- 
tained in the item or from the perspective of the computer screen; and 

• whether key words that have signs that might not be common to students or are directly 
related to the construct being measured should be signed only or signed and finger-spelled. 

These five issues are highlighted in this section of the report. Fourteen middle school Deaf/HH 
students who communicate using ASL participated in cognitive labs conducted for this study. 
Similar to their elementary school peers, most middle school students were observed to offer 
earnest responses to the test items, with some students appearing to guess some item responses. 

When asked whether viewing ASL videos helped them with understanding test questions bet- 
ter, most students responded affirmatively. Several students noted that being signed “makes it 
easier to understand.” One student shared, “Some of the English words I don’t know so I can 
watch the sign.” Another student stated, “Yes, absolutely! For sure! Some people might have 
difficulty with English. Being bilingual is difficult for some people.” 

Issue 1 : Do students prefer that items containing a iot of information prior 
to the question be signed using the Engiish text order (no-diamond) or by 
presenting the question before the item information and then again after the 
item information (diamond)? 

Two items were developed to answer whether students who are Deaf/HH prefer to have a signer 
present an item using the English text order (no-diamond) or present the question before the 
item information and after the item information (diamond). Table 8 shows student responses to 
questions about Item Set 1 . 

Replaying Video 

The majority of students did not replay the video for either item in Item Set I . However, one 
student replayed the first item in this set, where the no-diamond presentation was used. 
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Table 8. Student responses to questions about Item Set 1 


Types of Responses 

Item 1 

(No-dlamond) 

Item 2 
(Diamond) 

Replaying Video 

1 

0 

Difficulty 



Easy 

3 

4 

Medium 

2 

2 

Difficult 

6 

6 

No Reported Difficulty 

2 

1 

Item Preference 

4 

8 


Note: One student did not answer the items in item Set 1 . 


Reported Difficulty 

Those students who responded to the questions about item difficulty did not seem to reach 
consensus in their perceptions. The highest number of students, however, (n=6) perceived the 
two items to be difficult. 

Item Preferences 

Two-thirds of the respondents (n=8) reported that they preferred the diamond presentation, with 
the question presented both before and after the item information. One student who preferred 
this option stated, “The second item was better. You get the explanation twice, which is helpful.” 
Another student expressed a similar opinion, “I liked the second item. The repetition helped 
me understand it better.” Four other students preferred the no-diamond order of signing. One 
student also commented on the quality of the video presentations for this item set, “The signing 
speed is fine. The signs are big enough.” 

Issue 2: Do students prefer that information on number iines be presented by 
focusing on zero or by focusing on the number in the center of the number 
iine? 

Item Set 2 was developed to answer what perspective students prefer an item using a number 
line be explained from, with a focus on zero, or with a focus on the central number in the num- 
ber line. In the first question, the signer explained the number line by focusing on the number 
zero. In the second question, the signer explained the number line by focusing on the number 
two and one-fourth. Table 9 summarizes the findings for Item Set 2. 
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Table 9. Student responses to questions about Item Set 2 


Types of Responses 

Item 3 

(Number Line 
on Zero) 

Item 4 

(Number Line 
on Center) 

Replaying Video 

3 

1 

Difficulty 



Easy 

3 

4 

Medium 

2 

2 

Difficult 

6 

6 

No Reported Difficulty 

2 

1 

Item Preference 

7 

6 


Replaying Video 

As shown in the table, three students replayed the first item in Item Set 2 during the cognitive 
labs, and one student replayed the second item. 

Reported Difficulty 

The distribution of item difficulty perceptions shows that the highest number of respondents 
(n=6) believed that the two items were difficult, with lower numbers believing these items to 
be of medium difficulty or easy. Difficulty perceptions were not reported twice for the first item 
and once for the second item. 

Item Preferences 

Students appeared to be divided in their preferences for the two item presentations — seven 
students preferred the item that was signed with a focus on zero, and six students preferred the 
presentation with a focus on the central number in the number line. The majority of students also 
commented on the use of the drag-and-drop feature embedded in this item set. Many students 
reported using this feature either before assessments or in other contexts (“I have done drag- 
and-drop before — here and there.”). At least three respondents, however, reported that they had 
never come across this feature before. 

Issue 3: Do students prefer in items that require comparing expressions that 
the signer refer back to the originai expression or do not refer back to the 
originai expression? 

To answer this question, an isomorph item set was developed. In the first item, the signer did not 
refer back to the first expression when signing the expressions in the answer area. In the second 
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item, the signer referred back to the first expression when signing the expressions in the answer 
area. Student responses to questions about Item Set 3 are presented in Table 10. 


Table 10. Student responses to questions about Item Set 3 


Types of Responses 

Item 5 
(Do not 
Refer to 
Expression) 

Item 6 
(Refer to 
Expression) 

Replaying Video 

1 

1 

Difficulty 



Easy 

4 

4 

Medium 

3 

3 

Difficult 

5 

5 

No Reported Difficulty 

1 

1 

Item Preference 

3 

8 


Note: One student did not answer the questions in item Set 3. One student did not report a preference for either 
item in Item Set 3. 

Replaying Video 

For Item Set 3, each video was replayed once. 

Reported Difficulty 

The students who answered the questions about item difficulty varied in their responses, with 
three students considering these items to be of medium difficulty, four students believing that 
these items were easy, and five students stating that these items were difficult. In one case, item 
difficulty was not reported for each item. 

Item Preferences 

The majority of students (n=8) preferred the item in which the signer referred back to the origi- 
nal expression. A smaller number of students (n=3) preferred the item in which the signer did 
not refer back to the original expression. One student who preferred the order that is typically 
used in ASL noted, “I preferred the first item. The repetition made it easier to understand. I 
didn’t understand the second item.” Another student stated, “ASL is our language. English is 
not bad but ASL is my language. . . . When you read the English, it is hard to understand. But 
the signs help.” 

Issue 4: Do students prefer that items be presented in ASL from the signer’s 
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perspective of a diagram contained in the item or from the perspective of the 
computer screen? 

In the first item, the signer presented the rooms from the signer’s view of the house. In the second 
item, the signer presented the rooms from the view of the computer screen. Table 1 1 shows the 
student responses to questions about Item Set 4 at the middle school level. 


Table 1 1 . Student responses to questions about Item Set 4 


Types of Responses 

Item 7 
(Signer’s 
Perspective) 

Item 8 
(Computer 
Screen 
Perspective) 

Replaying Video 

1 

0 

Difficulty 



Easy 

8 

8 

Medium 

1 

1 

Difficult 

2 

2 

No Reported Difficulty 

2 

2 

Item Preference 

4 

7 


Note: One student did not answer the questions in item Set 4. One student did not report a preference for either 
item in Item Set 4. 

Replaying Video 

The table shows the number of students who replayed videos in Item Set 4. For this item set, 
one student replayed the first video only. 

Reported Difficulty 

The majority of students (n=8) considered the two items in Item Set 4 to be easy. The two items 
were also reported to be of medium difficulty once and of high difficulty twice. In two cases, 
this information was not reported. 

Item Preferences 

A greater number of students (n=7) expressed their preferences for the item in which the signer 
presented the rooms from the view of the computer screen (Item 8). Fewer students (n=4) pre- 
ferred the item in which the signer presented the rooms from his view of the house (Item 7). 
One of the students who preferred the item signed from the view of the computer screen stated, 
“The second video was easy to follow. The first one needs to be changed to be more ASL.” 
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Issue 5: Do students prefer that key words that have signs that might not be 
common to students or are directiy reiated to the construct being measured be 
signed oniy or signed and finger-speiied? 

An item set was developed to answer the researeh question of whether students prefer to have 
key words signed or finger-spelled and signed. In the first question, the signer used the ASL 
sign for “estimates.” In the second question, the signer used the ASL sign and finger- spelled 
“estimates.” Students’ responses to questions about Item Set 5 are shown in Table 12. 


Table 12. Student responses to questions about Item Set 5 


Types of Responses 

Item 9 
(Key Words 
Signed) 

Item 10 
(Key Words 
Signed and 
Finger- 
spelled) 

Replaying Video 

0 

0 

Difficulty 



Easy 

4 

4 

Medium 

2 

2 

Difficult 

2 

2 

No Reported Difficulty 

5 

5 

Item Preference 

5 

8 


Note: Two students did not answer the items in item Set 6. Three students did not report a preference for either 
item in Item Set 6. 

Replaying Video 

There were no students who replayed the signing video for Items 9 or 10. 

Reported Difficulty 

The questions about item difficulty were answered the following way: in two cases, the items 
were reported to be difficult; in two additional cases, the items were reported to be of medium 
difficulty; and in four cases, the items were reported to be easy. No difficulty was reported in 
five instances. 

Item Preferences 

Five students preferred the item in which “estimates” was signed only. Eight students preferred 
the item in which “estimates” was both signed and finger-spelled. The following are the com- 
ments from the students who preferred the item with the signed term only: 
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• “I knew the ASL sign for ‘estimates.’ The first video was easier to understand. The finger- 
spelling slows it down and is cumbersome.” 

• “Get rid of the finger-spelling. ASL is more clear. I am confused about where the finger- 
spelling was.” 

• “I tend to use ASL more. Having the word spelled out didn’t really help because I don’t 
know what it means, so spelling it wasn’t helpful.” 

Those who preferred the term both signed and finger-spelled stated the following: 

• “It is a good idea to have key terms finger- spelled. When they are spelled, I can learn and 
memorize them.” 

• “If I don’t know the term, it is helpful to have it finger- spelled.” 

• “I knew the sign for ‘estimates’ but it was helpful for me to have it finger- spelled.” 

• “Well, the math was difficult for me. I didn’t understand what the interpreter was talking 
about - the average and the numbers. I know the ASL sign for ‘estimates’ but it would be 
helpful to have it both finger-spelled and signed.” 

• “I didn’t know the term ‘estimates’ and I preferred to have it finger- spelled.” 


High School Assessment With Sign 

The high school assessment with sign support covered four item sets. Each item set was designed 
to present isomorph items with different presentation methods to discern which presentation 
method was most effective for students. The high school assessment covered the following four 
issues regarding how students preferred items be rendered: 

• whether key words that have signs that might not be common to students or are directly 
related to the construct being measured should be finger-spelled only, signed only, or both 
signed and finger-spelled; 

• whether expressions and equations should be presented in ASL; 

• whether items containing a lot of information prior to the question should be signed using 
the English text order (no-diamond) or by presenting the question before the item informa- 
tion and then again after the item information (diamond); and 
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• whether political cartoons should be fully described in ASL, or just the text within the po- 
litical cartoon should be signed. 

The four issues are described in this section of the report. Sixteen high school Deaf/HH students 
who communicate using ASL participated in cognitive labs conducted for this study. Most high 
school students were observed to offer earnest responses to the test items, with some students 
appearing to guess some responses. 

When asked whether watching ASL videos aided students with their understanding of test items, 
most students responded that such videos were helpful. One student noted: “Watching videos 
is helpful — if I don’t understand something, I can get more information.” 

Issue 1 : Do students prefer that key words that have signs that might not be 
common to students or are directiy reiated to the construct being measured be 
finger-speiied oniy, signed oniy, or signed and finger-speiied? 

A set of three items was developed to answer whether students who are Deaf/HH and com- 
municate using ASL prefer to have key words finger- spelled, signed only, or both signed and 
finger-spelled. In the first question, the signer finger-spelled the terms “diameter” and “circum- 
ference.” In the second question, the signer used ASL signs for those terms. In the third item, 
the signer signed the terms, then finger- spelled them. Table 13 summarizes student responses 
to questions about Item Set I. 

Table 13. Student responses to questions about Item Set 1 


Types of Responses 

Item 1 
(Finger- 
spelling 
Only) 

Item 2 
(ASL Only) 

Item 3 
(ASL and 
Finger- 
spelling) 

Replaying Video 

5 

2 

4 

Difficulty 




Easy 

0 

0 

0 

Medium 

4 

4 

3 

Difficult 

7 

7 

7 

No Reported Difficulty 

5 

5 

5 

Item Preference 

0 

8 

8 


Replaying Video 

In Item Set I, the first item was replayed five times, the second item was replayed twice, and the 
third item was replayed four times. One student who replayed the second item noted, “I liked 
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the second video... I watched it twice because I was thrown off by the finger- spelling — the 
same word was presented more than once.” 

Reported Difficulty 

The majority of students (n=7) reported that the three items in Item Set 1 were difficult. In five 
cases, the item difficulty information was not collected. 

Item Preferences 

Students were evenly divided between preferring the item with key words signed only (n=8) 
and the item with key words both signed and finger- spelled (n=8). No students expressed their 
preference for the item in which key words were finger-spelled only. 

One of the students who preferred the sign-only video stated, “It is easier to remember by just 
having ASL signs. I knew the signs for ‘diameter’ and ‘circumference.’ Finger- spelling is confus- 
ing.” Another student with this preference noted, “I preferred the second video - with just ASL. 
It was clear. I understood the concepts. I knew the signs for ‘diameter’ and ‘circumference.’” 

The following three comments come from the students who preferred the terms signed and 
finger-spelled: 

• “The third video with both was most helpful. It is clearer - I know what it’s related to. 
Vocabulary and finger-spelling are helpful. I knew the ASL signs for ‘circumference’ and 
‘diameter.’” 

• “I liked both the sign and finger- spelling. If you have the sign, the spelling goes with it. I 
didn’t know the signs for the two terms.” 

• “I preferred signing with finger-spelling. When the word matches the sign, it is most under- 
standable. I knew the sign for ‘diameter’ but for ‘circumference,’ we have a different sign.” 

Issue 2: Do students prefer that expressions and equations be presented in 
ASL? 

Item Set 2 was developed to determine whether students prefer expressions and equations be 
presented in ASL. In the first question, the equations were signed. In the second question, the 
equations were not signed. Student responses to questions about Item Set 2 are presented in 
Table 14. 
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Table 14. Student responses to questions about Item Set 2 


Types of Responses 

Item 4 
(Equations 
Signed) 

Item 5 
(Equations 
not Signed) 

Replaying Video 

1 

0 

Difficulty 



Easy 

2 

2 

Medium 

3 

3 

Difficult 

7 

7 

No Reported Difficulty 

4 

4 

Item Preference 

12 

2 


Note: Two students did not answer the items in item Set 2. 


Replaying Video 

In one case, Item 4 in this set, where equations were signed, was replayed. 

Reported Difficulty 

In most cases (n=7), the two items in Item Set 2 were perceived by students to be difficult. No 
difficulty was reported in four cases. 

Item Preferences 

Most respondents (n=12) preferred that expressions and equations be signed, while only two 
respondents preferred the items in which expressions and equations were not signed. Those 
students who preferred the item with signed expressions and equations pointed out that the 
item was “clearer,” and they were able to get both “the directions and the equation.” Another 
student stated: “The first video is more helpful — it is more understandable, says more. I don’t 
know what the words mean otherwise.” Still another student wondered, “I liked the one with 
the equation signed. Because otherwise, what are you talking about?” 

Issue 3: Do students prefer that items containing a iot of information prior 
to the question be signed using the Engiish text order (no-diamond) or by 
presenting the question before the item information and then again after the 
item information (diamond)? 

To answer whether students prefer that expressions and equations be presented in ASL, an item 
set was developed. In the first question, the signer presented the question twice, once at the 
beginning of the item and again after the item information. In the second question, the signer 
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presented the information in the order it appears in English. Table 15 shows students’ responses 
to questions about Item Set 3. 


Table 15. Student responses to questions about Item Set 3 


Types of Responses 

Item 6 
(Diamond) 

Item 7 

(No-dlamond) 

Replaying Video 

7 

4 

Difficulty 



Easy 

2 

2 

Medium 

0 

0 

Difficult 

6 

5 

No Reported Difficulty 

8 

9 

Item Preference 

11 

0 


Note: Two students did not answer the items in item Set 3. Three students did not report a preference for either 
item in Item Set 3. 

Replaying Video 

In Item Set 3, the first item (diamond presentation) was replayed by seven students, and the 
second item (no-diamond presentation) was replayed by four students. 

Reported Difficulty 

Of those who reported on this aspect, six students perceived the first item to be difficult, and 
five students perceived the second item to be difficult. Eight students did not indicate difficulty 
for Item 8 and nine did not for Item 9. 

Item Preferences 

All of those respondents who addressed the questions about their item preferences (n=ll) 
reported that they preferred the item with the diamond structure, with the question presented 
both at the beginning and the end of the item because this item was “clearer.” According to one 
student, in the first video, “the interpreter signed the question again, so I could remember it.” 
According to another student, “The first item was more helpful — question-problem-question — so 
you know if you are not sure what the question was, it gets repeated.” A third student noted: “I 
preferred the first item — it was more clearly explained. The English order is too much. English 
was confusing, I watched it twice. ASE is more condensed, easy.” 
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Issue 4: Do students prefer that political cartoons be described in ASL? 

To answer the question posed by Issue 4, an isomorph set of items was developed. In the first item, 
the political cartoon was described by the signer. In the second item, only the text in the politi- 
cal cartoon was signed. Student responses to questions about Item Set 4 are shown in Table 16. 

Table 16. Student responses to questions about Item Set 4 

Item 8 Item 9 

Types of Responses (Cartoon (Cartoon not 

Described) Described) 


Replaying Video 5 5 

Difficulty 



Note: One student did not answer the items in Item Set 5. 


Replaying Video 

Both items in Item Set 4 were replayed in five cases. 

Reported Difficulty 

There appears to be no consensus among students when it comes to their perceptions of item dif- 
ficulty. The responses are almost evenly distributed along the easy-medium-difficult continuum. 
No data were reported for the first item in seven cases and for the second item in six cases. 

Item Preferences 

The majority of high school students (n=13) preferred the item in which the political cartoon was 
described. Only two students preferred the item in which the signer did not describe the political 
cartoon. The following comments illustrate students’ preferences for the first item in Item Set 4: 

• “I liked the one with the picture described. In Deaf culture, we do describe photos.” 

• “I preferred the first video. The first one was, like, that’s it! The first one had more explana- 
tions.” 
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• “I liked the picture described. The signer explained the picture, how you could see it, what’s 
going on.” 

• “I liked when the interpreter described the picture because he provided all the details.” 


Additional Student Characteristics • 

Students were also asked additional background questions at the end of each cognitive lab. 
These questions related to the length of time they have used ASL and family members who 
were deaf or hard of hearing. 

When asked about how long students had been using ASL, several of them responded that they 
had used it as long as they could remember. Other students noted that they learned ASL when 
they began school. A few students reported that they had only been learning ASL for about a 
year. It should be noted that a small number of students had relocated from another country in 
order to attend their current school. In these cases, students mentioned that they had previously 
learned a different sign language or they had not been previously exposed to ASL. 

Students were also asked about whether they had family members who were deaf or hard of 
hearing. Of the 46 students who participated in the study, 16 mentioned having at least one 
family member who is deaf. 


Discussion 

This study sought to gain student feedback and preference regarding how assessment items 
can be best presented using ASL. The findings from this study are generally mixed, pointing 
to the need for assessments to take an individualized approach to assigning accommodations 
for students who are Deaf/HH. Although standardization of video presentations is a critical 
consideration, it is also important to be mindful of embedding such options as additional finger- 
spelling, for example. It is also important to note that students who participated in this study 
ranged in the number of years that they were exposed to ASL, and their proficiency in ASL 
may have been a factor in their preferences. It should also be noted that ASL, similar to other 
languages, is characterized by variations, and some students’ understanding of the videos may 
have been affected by such variations. 

One key issue this study addressed was the structure of the item presentations in ASL. Given 
that ASL is a different language from English, it is not surprising that the structure of ASL is 
also different. In translating content from English into ASE, there have been different schools 
of thought as to whether English or ASE structures should be used. In this study, one example 
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of this difference is the items that follow the diamond pattern. In these item sets, one item is 
presented in the order it appears in English, or in the no-diamond pattern. In the other item, 
the question is presented first, then the information from the problem comes next, followed by 
a repeat of the question. This is referred to as the diamond pattern. Items addressing this issue 
were presented in both middle and high school (Issue I in middle school, and Issue 3 in high 
school). In both grade bands, students overwhelmingly preferred the diamond presentation over 
the no-diamond alternative. 

In some cases, students were mixed in other aspects of items that followed ASL conventions. 
High school students preferred to have political cartoons described in ASL. Students did not 
have a strong preference about whether the signer represented coordinate grids on his hand or 
in front of his body. They were also divided on whether the signer should represent space from 
the perspective of the signer or the computer screen. 

Another issue that was examined across all grade levels was finger-spelling. This issue is im- 
portant because some mathematical terms have no specific ASL equivalent or have multiple 
equivalents, and although there are general agreements on some terms, local signs may also be 
used. Linger-spelling can be a means to standardize the presentation of these terms; however, 
finger- spelling poses additional challenges as well. Students were given items in which math 
terms were signed and other items in which math terms were finger-spelled. As with other items, 
students were mixed in whether they believed finger- spelling was helpful or distracting. In some 
cases, students knew the ASL signs for math terms; in other cases they did not. However, some 
students also did not know what the terms were, even when they were finger-spelled. Addition- 
ally, some students reported that having the items finger- spelled was helpful. This type of items 
is further evidence that having multiple means of representation, in order to allow individualized 
approaches to accessing the item content, may be the best approach. 

Limitations of the Study 

There are a few important considerations with regard to the data presented here. One limitation 
to the study involves the sample of students who participated in the cognitive labs. All students 
participating in the signed cognitive labs were enrolled in schools for students who are Deaf. 
Although attempts were made to recruit students who are enrolled in mainstream schools, only 
students enrolled in schools for students who are Deaf participated in the study. 

Another limitation of the study related to students’ math abilities. In some cases, students admit- 
ted that math was a challenging subject for them, and in relation to specific questions, students 
noted that they had not been taught the content of the questions. In other cases, students who 
were more confident in math often skipped the ASL videos because they did not see a need to 
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watch those videos. In these cases, the researchers asked the students to watch the videos prior 
to attempting to solve the problems. 

One question in the study had a limitation as well. In the elementary school items, in Issue 2, 
the signer asks students to drag corn into the box shown. In this item, students were observed 
to take the signer literally: he signs dragging the corn three times. Most students then were ob- 
served to drag three kernels of corn into the box. This may suggest that the alternative signed 
approach was more appropriate; however, additional research may be beneficial before ruling 
out this approach to signing this type of question. 

Conclusion 

The sign support cognitive labs administered as part of the Guidelines for Accessible Assess- 
ments Project were conducted with elementary, middle, and high school Deaf/HH students. 
The research results point to the students’ preferences for the provision of the sign support in 
general and for this support being grounded in ASL rather than representations aligned with 
other languages. Overall, students also voiced their preferences for graphics to be described in 
detailed ASL. Yet, in spite of these tendencies, the results were mixed overall. These findings 
support the current shift in the assessment paradigm — from making blanket decisions based on 
a student category to making individualized decisions informed by students’ unique needs and 
preferences. When developing sign supports for computer-based assessments, it is important to 
consider various needs of the diverse community of Deaf/HH students. 
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Appendix A: Study Instruments 


School #: 

Student #: 

Grade {circle one): 3 4 5 

Need {circle all that apply): Deaf/Hard of Hearing Print Disability 

English Learner Low Vision 

Student Gender {circle one): Male Lemale 

Test Login ID: 

Note Taker: 

Date: 


Grades 3-5 Sign 

Thank you for participating in this study to understand more about how test questions are 
presented in American Sign Language. ~We would like for you to view and answer 12 questions 
on the computer. Each question will be signed to you on a video screen that also contains the 
item in English. Press the play button to view the video, you can view all or parts of the item as 
many times as you like. We will ask you questions about what you are thinking as you answer 
the test questions. 

Item Set One: Coordinate Grid/Use of Space 
Observation Notes 


After #I & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 
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Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 


After the set: 

The two questions that you just viewed were presented differently. In the first question, the 
signer presented the graph on his hand. In the second question, the signer presented the graph 
in front of his body. Which way do you think helped you answer the question? Did you like one 
way better than the other? If so, why did you like one of them better? 


Item Set Two: Drag and Drop 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 
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Is there anything that you would change to make the sign video better? Please explain what 
that would be. 


After the set: 

The two questions that you just viewed were presented differently. In the first question, the 
signer presented the question from the view of the farmer, in front of his body, using top to bot- 
tom space, and asked you to be the farmer and plant the soybeans. In the second question, the 
signer presented the question from the view of the computer screen, using the side-to-side space. 
Which way do you think helped you answer the question ? Did you like one way better than the 
other? If so, why did you like one of them better? 


Item Set Three: Equations 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 
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Is there anything that you would change to make the sign video better? Please explain what 
that would be. 


After the set: 

The two questions that you just viewed were presented differently. In the first question, the equa- 
tions were not signed. In the second question, the equations were signed. Which way do you 
think helped you answer the question? Did you like one way better than the other? If so, why 
did you like one of them better? 


Item Set Four: Fingerspelling 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you just viewed were presented differently. In the first question, the signer 
used the ASL signs for right angle, symmetry, and length. In the second question, the signer used 
ASL signs and finger-spelled those terms. Did you know the signs for right angle, symmetry, and 
length.^ Was it helpful to have these words finger-spelled or did that make it more confusing? 
Which way do you think helped you answer the question ? Did you like one way better than the 
other? If so, why did you like one of them better? 


Item Set Five: Plurality and Repetition 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you just viewed were presented differently. In the first question, the 
signer showed muffins being put in a box one time. In the second question, the signer showed 
cookies being put into multiple bags. Which way do you think helped you answer the question ? 
Did you like one way better than the other? If so, why did you like one of them better? In the 
second question, the word bags was finger-spelled in the answer options. Was it helpful to have 
the word bags finger-spelled or did that make it more confusing? 


Item Set Six: Signing Answer Options 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 

Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you just viewed were presented differently. In the first question, the 
numbers in the answer choices were signed to you. In the second question, the numbers in the 
answer choices were not signed to you. Which way do you think helped you answer the ques- 
tion? Did you like one way better than the other? If so, why did you like one of them better? 
In the first question, the word pages was finger-spelled in the answer options. Was it helpful to 
have the word pages finger-spelled or did that make it more confusing? In the second question, 
the word miles was finger-spelled in the answer options. Was it helpful to have the word miles 
finger-spelled or did that make it more confusing? 


After the Sign Test: 

For how many years have you communicated in ASL? 


Are other members of your family deaf or hard of hearing? 


At what age was the onset of your hearing loss? 


Do you have hearing loss in one or both ears? 


What caused your deafness/hearing loss (etiology)? 


What is your decibel loss? 


Do you wear cochlear implants? 


Do you view ASL videos in the classroom (outside of testing)? 
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When taking this practice test, did you usually read the item first or view the ASL video first? 


Do you believe that viewing ASL videos helps you understand test questions better? 
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School #: 

Student #: 

Grade {circle one): 6 7 8 

Need {circle all that apply): Deaf/Hard of Hearing Print Disability 

English Learner Low Vision 

Student Gender {circle one): Male Lemale 

Test Login ID: 

Note Taker: 

Date: 


Grades 6-8 Sign 

Thank you for participating in this study to understand more about how test questions are 
presented in American Sign Language. We would like for you to view and answer 12 questions 
on the computer. Each question will be signed to you on a video screen that also contains the 
item in English. Press the play button to view the video, you can view all or parts of the item as 
many times as you like. We will ask you questions about what you are thinking as you answer 
the test questions. 


Item Set One: Coordinate Grid/Diamond Presentation 
Observation Notes: 


After #I & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 
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Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 


After the set: 

The two questions that you viewed were presented differently. In the first question, the signer 
presented the information in the order it appears in English. In the second question, the signer 
presented the question twice, once at the beginning of the item and again after the graph. Which 
way did you think helped you answer the question ? Did you like one way better than the other? 
If so, why did you like one of them better? 


Item Set Two: Drag and Drop/Presentation of number line 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 
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Is there anything that you would change to make the sign video better? Please explain what 
that would be. 


After the set: 

The two questions that you viewed were presented differently. In the first question, the signer 
explained the number line by focusing on the number zero. In the second question, the signer 
explained the number line by focusing on the number 2 and one-fourth. Which way did you 
think helped you answer the question? Did you like one way better than the other? If so, why 
did you like one of them better? 


Item Set Three: Equations 
After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you viewed were presented differently. In the first question, the signer 
did not refer back to the first expression when signing the expressions in the answer area. In the 
second question, the signer referred back to the first expression when signing the expressions 
in the answer area. Which way did you think helped you answer the question ? Did you like one 
way better than the other? If so, why did you like one of them better? 


Item Set Four: Diagram/Point of View 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you viewed were presented differently. In the first question, the signer 
presented the rooms from his view of the house. In the second question, the signer presented the 
rooms from the view of the computer screen. Which way did you think helped you answer the 
question ? Did you like one way better than the other? If so, why did you like one of them better? 


Item Set Five: Finger spelling 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you viewed were presented differently. In the first question, the signer used 
the ASL sign for estimates. In the second question, the signer used ASL signs and fingerspells 
estimates. Did you know the sign for estimates ? Was it helpful to have these words finger-spelled 
or did that make it more confusing ? 


After the Sign Test: 

For how many years have you communicated in ASL? 


Are members of your family deaf or hard of hearing? 


At what age was the onset of your hearing loss? 


Do you have hearing loss in one or both ears ? 


What caused your deafness/hearing loss (etiology)? 


What is your decibel loss? 


Do you wear cochlear implants? 


Do you view ASL videos in the classroom (outside of testing)? 


When taking this practice test, did you usually read the item first or view the ASL video first? 


Do you believe that viewing ASL videos helps you understand test questions better? 
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School #: 

Student #: 

Grade (drc/e one): 9 10 11 12 

Need {circle all that apply): Deaf/Hard of Hearing Print Disability 

English Learner Low Vision 

Student Gender {circle one): Male Lemale 

Test Login ID: 

Note Taker: 

Date: 


Grades 9-12 Sign 

Thank you for participating in this study to understand more about how test questions are 
presented in American Sign Language. We would like for you to view and answer 9 questions 
on the computer. Each question will be signed to you on a video screen that also contains the 
item in English. Press the play button to view the video, you can view all or parts of the item as 
many times as you like. We will ask you questions about what you are thinking as you answer 
the test questions. 

Item Set One: Linger spelling 
Observation Notes: 


After #1, #2, & #3: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 
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Is there anything that you would change to make the sign video better? Please explain what 
that would be. 


After the set: 

The two questions that you just viewed were presented differently. In the first question, the signer 
finger-spelled the terms diameter and circumference. In the second question, the signer used 
ASL signs for those terms. In the third item, the signer signed the terms, then finger-spelled 
them. Did you know the sign for diameter and circumference ? Was it helpful to have these words 
finger-spelled or did that make it more confusing? 

Item Set Two: Equations 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 

Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you just viewed were presented differently. In the first question, the 
equation was signed. In the second question, the equation was not signed. Which way did you 
think helped you answer the question? Did you like one way better than the other? If so, why 
did you like one of them better? 


Item Set Three: Diamond Presentation 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you just viewed were presented differently. In the first question, the signer 
presented the question twice, once at the beginning of the item and again after the money infor- 
mation. In the second question, the signer presented the information in the order it appears in 
English. Which way did you think helped you answer the question ? Did you like one way better 
than the other? If so, why did you like one of them better? 


Item Set Four: Political Cartoons 
Observation Notes: 


After #1 & #2: 

Was the way this question was signed to you helpful? Why or why not? 


If necessary, probe: 

Was there anything/anything else about how this question was signed that was helpful? Can 
you please explain how the video was helpful? 


Was there anything/anything else about how this question was signed that was not helpful? Can 
you please explain how the read aloud was not helpful? 


Was anything/anything else difficult to understand in the video ? Please explain what that was. 


Is there anything that you would change to make the sign video better? Please explain what 
that would be. 
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After the set: 


The two questions that you just viewed were presented differently. In the first question, the signer 
described the picture and the text. In the second question, the signer did not describe the picture. 
Which way did you think helped you answer the question ? Did you like one way better than the 
other? If so, why did you like one of them better? 

After the Sign Test: 

For how many years have you communicated in ASL? 


Are members of your family deaf or hard of hearing? 


At what age was the onset of your hearing loss? 


Do you have hearing loss in one or both ears? 


What caused your deafness/hearing loss (etiology)? 


What is your decibel loss? 


Do you wear cochlear implants? 


Do you view ASL videos in the classroom (outside of testing)? 


When taking this practice test, did you usually read the item first or view the ASL video first? 


Do you believe that viewing ASL videos helps you understand test questions better? 
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